Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Optimizer: Add partition by support for derived TopN(filter on row_number) #41362

Closed
wants to merge 35 commits into from
Closed

Optimizer: Add partition by support for derived TopN(filter on row_number) #41362

wants to merge 35 commits into from

Conversation

ghazalfamilyusa
Copy link
Contributor

Issue Number: Ref #39792

Problem Summary:

This is a continuation of #41209 which introduced deriving top N from "row_number as RN ... where RN <= N". This PR adds support for partition by in the window function "row_number as RN over (partition by X) ... where RN <= N" and restricted if the partition by is a prefix of the PK. Execution support for this extension is here tikv/tikv#14116

Tests

Unit tests

Side effects

None

Documentation

None

Release note

None

@ti-chi-bot
Copy link
Member

[REVIEW NOTIFICATION]

This pull request has not been approved.

To complete the pull request process, please ask the reviewers in the list to review by filling /cc @reviewer in the comment.
After your PR has acquired the required number of LGTMs, you can assign this pull request to the committer in the list by filling /assign @committer in the comment to help you merge this pull request.

The full list of commands accepted by this bot can be found here.

Reviewer can indicate their review by submitting an approval review.
Reviewer can cancel approval by submitting a request changes review.

@ti-chi-bot ti-chi-bot added release-note-none Denotes a PR that doesn't merit a release note. size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Feb 13, 2023
@ghazalfamilyusa ghazalfamilyusa marked this pull request as draft February 14, 2023 17:31
@ti-chi-bot ti-chi-bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 14, 2023
@ghazalfamilyusa ghazalfamilyusa force-pushed the derived_topn_with_partition branch 2 times, most recently from 449ea8f to 5944195 Compare February 14, 2023 23:51
@ghazalfamilyusa ghazalfamilyusa marked this pull request as ready for review February 14, 2023 23:51
@ti-chi-bot ti-chi-bot removed the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 14, 2023
@ghazalfamilyusa ghazalfamilyusa force-pushed the derived_topn_with_partition branch 5 times, most recently from d9974ca to 6de55f1 Compare February 15, 2023 20:54
@ti-chi-bot ti-chi-bot added the needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. label Feb 15, 2023
@ti-chi-bot
Copy link
Member

@ghazalfamilyusa: PR needs rebase.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository.

@ti-chi-bot ti-chi-bot added size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files. and removed size/XL Denotes a PR that changes 500-999 lines, ignoring generated files. labels Feb 15, 2023
@ghazalfamilyusa ghazalfamilyusa force-pushed the derived_topn_with_partition branch from 6de55f1 to a10b52c Compare February 15, 2023 21:05
ghazalfamilyusa and others added 10 commits February 15, 2023 13:23
@ghazalfamilyusa
Delete diff
749852b
@ystaticy @ghazalfamilyusa
keyspace: add keyspace util func and ut (pingcap#41353)
ca25b82 
ref pingcap#41354
@time-and-fate @ghazalfamilyusa
planner: stabilize tests in binary_plan_test.go (pingcap#41356)
61498d1 
close pingcap#40004, close pingcap#41297
@qw4990 @ghazalfamilyusa
planner: small refactor about plan cache checker (pingcap#41357)
5dd6f12 
ref pingcap#36598
@hawkingrei @ghazalfamilyusa
*: update bazel (pingcap#41369)
aa28289 
ref pingcap#41377
@AstroProfundis @ghazalfamilyusa
telemetry: set switch default to false (pingcap#41336)
c99c44c 
close pingcap#41335
@Defined2014 @ghazalfamilyusa
etcd_test: remove unnecessary sleep, make ci more stable (pingcap#41378)
30037c0 
close pingcap#37652
@xiabee @ghazalfamilyusa
br: security update br image:1.18.1-alpine to 1.18.9-alpine (pingcap#…
eda4480 
…40863)

ref pingcap#39052
@xuyifangreeneyes @ghazalfamilyusa
domain, tidb-server: stop launching new auto analyze job when shuttin…
d649179 
…g down (pingcap#41346)

close pingcap#41318
@time-and-fate @ghazalfamilyusa
planner: add missing CanExprsPushDown checks when generating IndexM…
e7a302f 
…erge path for `or` (pingcap#41361)

close pingcap#41273, close pingcap#41293
ystaticy and others added 24 commits February 15, 2023 13:33
@ystaticy @ghazalfamilyusa
keyspace: fix keyspace ut (pingcap#41375)
1022586 
close pingcap#41376
@hawkingrei @ghazalfamilyusa
*: improve makefile (pingcap#41399)
c5fe1eb
@fengou1 @ghazalfamilyusa
br: reused table id is disabled when restore a brand-new cluster (pin…
0231621 
…gcap#41358)

close pingcap#41333
@tangenta @ghazalfamilyusa
ddl: set request source for cop requests that are used in adding index (
2a43ad6 
pingcap#41401)

close pingcap#41400
@zhongzc @ghazalfamilyusa
stmtv2: ignore corrupted line when scan (pingcap#41374)
27136fe 
close pingcap#41373
@hawkingrei @ghazalfamilyusa
*: upgrade golangci-linter (pingcap#41317)
ad3c1b9
@hawkingrei @ghazalfamilyusa
statistics: improve test to make test more fast (pingcap#41403)
2067b5d 
ref pingcap#41377
@qw4990 @ghazalfamilyusa
planner: add more test cases for non-prepared plan cache (pingcap#41398)
b290b0b 
ref pingcap#36598
@zimulala @ghazalfamilyusa
ddl: fix cleanup subtasks after cancel add index with dist-reorg (pin…
d9dc46c 
…gcap#41372)

close pingcap#41371
@hawkingrei @ghazalfamilyusa
*: bazel upgrade shard_count (pingcap#41417)
652641d
@you06 @ghazalfamilyusa
executor: fix race when collecting telemetry info of batch copr (ping…
663478d 
…cap#41413)

close pingcap#41412
@hawkingrei @ghazalfamilyusa
expression: improve expression to make test faster (pingcap#41426)
87f8b3b 
ref pingcap#41377
@zimulala @ghazalfamilyusa
ddl: make reorgCtx safer to use (pingcap#41421)
e87e9ee 
close pingcap#41418
@tangenta @ghazalfamilyusa
ddl: ignore reorg elem error when the job is cancelling (pingcap#41383)
4c0107f 
close pingcap#41381
@hawkingrei @ghazalfamilyusa
*: improve bazel config (pingcap#41430)
33251f4
@crazycs520 @ghazalfamilyusa
session: enable foreign_key_checks/tidb_enable_foreign_key by def…
c2a7146 
…ault when upgrade (pingcap#41424)
@lance6716 @ghazalfamilyusa
parser: add NULL DEFINED BY and FORMAT to LOAD DATA (pingcap#41332)
f7b036d 
ref pingcap#40499
@qw4990 @ghazalfamilyusa
planner: add more test cases for non-prepared plan cache (pingcap#41451)
a7ca870 
ref pingcap#36598
@WangLe1321 @ghazalfamilyusa
br: open parallel test for storage_test (pingcap#41450)
f7bb8b2 
close pingcap#41449
@hawkingrei @ghazalfamilyusa
*: improve executor to make test faster (pingcap#41442)
1f9f8c0 
close pingcap#41377
@zimulala @ghazalfamilyusa
docs: add a proposal for distributed DDL reorg (pingcap#41256)
5eed62c 
close pingcap#37129
@AntiTopQuark @ghazalfamilyusa
expression: implement is_ipv4 and is_ipv6 function pushdown (pingcap#…
2de1d17 
…41173)

close pingcap#41172
@hawkingrei @ghazalfamilyusa
*: improve planner/core to make test faster (pingcap#41416)
2f9e7b2 
ref pingcap#41377
@Dousir9 @ghazalfamilyusa
planner: add more checks when pushing TopN down (pingcap#41370)
68a61de 
close pingcap#41355
@ghazalfamilyusa ghazalfamilyusa requested a review from a team as a code owner February 15, 2023 22:51
@ghazalfamilyusa ghazalfamilyusa marked this pull request as draft February 15, 2023 22:52
@ti-chi-bot ti-chi-bot added the do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. label Feb 15, 2023
@ti-chi-bot
Copy link
Member

@ghazalfamilyusa: The following tests failed, say /retest to rerun all failed tests or /retest-required to rerun all mandatory failed tests:

Test name Commit Details Required Rerun command
idc-jenkins-ci-tidb/unit-test 68a61de link true /test unit-test
idc-jenkins-ci-tidb/mysql-test 68a61de link true /test mysql-test
idc-jenkins-ci-tidb/check_dev 68a61de link true /test check-dev
idc-jenkins-ci-tidb/build 68a61de link true /test build
idc-jenkins-ci-tidb/check_dev_2 68a61de link true /test check-dev2

Full PR test history. Your PR dashboard.

Instructions for interacting with me using PR comments are available here. If you have questions or suggestions related to my behavior, please file an issue against the kubernetes/test-infra repository. I understand the commands that are listed here.

@ghazalfamilyusa ghazalfamilyusa deleted the derived_topn_with_partition branch February 16, 2023 00:13
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Reviewers
No reviews
Assignees
No one assigned
Labels
do-not-merge/work-in-progress Indicates that a PR should not merge because it is a work in progress. needs-rebase Indicates a PR cannot be merged because it has merge conflicts with HEAD. release-note-none Denotes a PR that doesn't merit a release note. size/XXL Denotes a PR that changes 1000+ lines, ignoring generated files.
Projects
None yet
Milestone
No milestone
Development

Successfully merging this pull request may close these issues.